Reconstructor and contrastor for anomaly detection

ABSTRACT

Systems and methods for detecting and correcting defective products include capturing at least one image of a product with at least one image sensor to generate an original image of the product. An encoder encodes portions of an image extracted from the original image to generate feature space vectors. A decoder decodes the feature space vectors to reconstruct the portions of the image into reconstructed portions by predicting defect-free structural features in each of the portions according to hidden layers trained to predict defect-free products. Each of the reconstructed portions are merged into a reconstructed image of a defect-free representation of the product. The reconstructed image is communicated to a contrastor to detect anomalies indicating defects in the product.

RELATED APPLICATION INFORMATION

This application claims priority to 62/525,291, filed on Jun. 27, 2017,incorporated herein by reference in its entirety. This application isrelated to an application entitled “RECONSTRUCTOR AND CONTRASTOR SYSTEMFOR MEDICAL ANOMALY DETECTION”, having attorney docket number 17029B,and which is incorporated by reference herein in its entirety.

BACKGROUND Technical Field

The present invention relates to anomaly detection and more particularlyreconstructors and contrastors for anomaly detection applications.

Description of the Related Art

Defects and anomalies appear in products prior to shipment of thoseproducts to customers. The defects and anomalies can occur in severalways, in varying frequency and of diverse types depending on the productin question and how it was made. However, the defective productsgenerally do not meet a standard to satisfy a customer due to thedefects and anomalies.

Thus, removing the products from a shipment can prevent customerdissatisfaction and ensure a consistent product quality. However,recognizing which products contain defects can be difficult becauseexamples of defective products, especially products with particulartypes of defects, can be relatively rare.

SUMMARY

According to an aspect of the present principles, a method is providedfor detecting and correcting defective products. The method includescapturing at least one image of a product with at least one image sensorto generate an original image of the product. An encoder encodes each ofat least one portion of an image extracted from the original image togenerate a feature space vector. A decoder decodes the feature spacevector to reconstruct the at least one portion of the image into acorresponding at least one reconstructed portion by predictingdefect-free structural features in each of the at least one portionaccording to hidden layers trained to predict defect-free products. Eachof the at least one reconstructed portion are merged into areconstructed image of a defect-free representation of the product. Thereconstructed image is communicated to a contrastor to detect anomaliesindicating defects in the product.

According to another aspect of the present principles, a method isprovided for detecting and correcting defective products. The methodincludes extracting, with an image patch extractor, portions of anoriginal image of a product. The original image is reconstructed with areconstructor, including: encoding, with an encoder, each of at leastone portion of an image extracted from the original image to generate afeature space vector, and decoding, with a decoder, the feature spacevector to reconstruct the at least one portion of the image into acorresponding at least one reconstructed portion by predictingdefect-free structural features in each of the at least one portionaccording to hidden layers trained to predict defect-free products. Theportions into the original image are merged with an image merging moduleto generate a reconstructed image. The original image of the product iscontrasted contrasting, with a contrastor, with the reconstructed imageto generate an anomaly map indicating locations of difference betweenthe original image and the reconstructed image. The locations ofdifference are tagged as anomalies with an anomaly tagging devicecorresponding to defects on the product. An operator is notifiedautomatically with a display of the anomalies.

According to another aspect of the present principles, a system isprovided for detecting and correcting defective products. The systemincludes at least one image sensor for capturing at least one image of aproduct to generate an original image of the product. A reconstructorreconstructs an original image of a product by reconstructing portionsof the original image to be a defectless representation of the portionsof the product, the reconstructor including: an encoder to encode eachof at least one portion of an image extracted from the original image togenerate a feature space vector, and a decoder to decode the featurespace vector to reconstruct the at least one portion of the image into acorresponding at least one reconstructed portion by predictingdefect-free structural features in each of the at least one portionaccording to hidden layers trained to predict defect-free products. Animage merging module merges each of the at least one reconstructedportion into a reconstructed image of a defect-free representation ofthe product. An anomaly correction system automatically discards theproduct according to the tagged anomalies.

These and other features and advantages will become apparent from thefollowing detailed description of illustrative embodiments thereof,which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

The disclosure will provide details in the following description ofpreferred embodiments with reference to the following figures wherein:

FIG. 1 is a block/flow diagram illustrating a high-level system/methodfor correcting defective products, in accordance with the presentprinciples;

FIG. 2 is a block/flow diagram illustrating a system/method fordetecting and correcting defects and anomalies, in accordance with thepresent principles;

FIG. 3 is a block/flow diagram illustrating a system/method fordetecting and correcting defects and anomalies using a reconstructor andcontrastor, in accordance with the present principles;

FIG. 4 is a block/flow diagram illustrating a system/method forreconstructing an image for detecting and correcting defects andanomalies, in accordance with the present principles;

FIG. 5 is a block/flow diagram illustrating a system/method forcontrasting a reconstructed image and original image for detecting andcorrecting defects and anomalies, in accordance with the presentprinciples;

FIG. 6 is a block/flow diagram illustrating a system/method for traininga reconstructor for detecting and correcting defects and anomalies, inaccordance with the present principles;

FIG. 7 is a block/flow diagram illustrating a high-level system/methodfor diagnosing medical anomalies, in accordance with the presentprinciples; and

FIG. 8 is a flow diagram illustrating a system/method for detectingdefects and anomalies with a reconstructor and contrastor, in accordancewith the present principles.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In accordance with the present principles, systems and methods areprovided for anomaly detecting. In particularly useful embodiments,systems and methods can be configured to correct defects in products,such as, e.g., manufactured products.

In one embodiment, a product is inspected for defects using areconstructor and contrastor. The reconstructor can include anencoder-decoder arrangement employed to reconstruct images of a productcontaining defects. The encoder-decoder arrangement is trained fromexamples that contain no defects. Thus, the encoder-decoder arrangementwill use an incomplete image of the product, and reconstruct the imageaccording to the defectless training. As a result, a reconstructed imageof the product can be produced where the reconstructed image depicts theproduct having no defects.

The contrastor can then determine a difference between the reconstructedimage and an original image of the product. If there is a substantialdifference between the original image and the reconstructed image, thenthe product is considered not defectless. In other words, by failing tomatch a defectless reconstructed image of the product, it can bedetermined that the product does contain some anomaly or defect.

Because the systems and methods employed to determine if a product failsto match a defectless reconstruction, an encoder-decoder arrangement canbe trained with only normal, defectless images of the product. Thus,training samples can be found in abundance, and thus training theencoder-decoder arrangement can be cheap and efficient.

Embodiments described herein may be entirely hardware, entirely softwareor including both hardware and software elements. In a preferredembodiment, the present invention is implemented in software, whichincludes but is not limited to firmware, resident software, microcode,etc.

Embodiments may include a computer program product accessible from acomputer-usable or computer-readable medium providing program code foruse by or in connection with a computer or any instruction executionsystem. A computer-usable or computer readable medium may include anyapparatus that stores, communicates, propagates, or transports theprogram for use by or in connection with the instruction executionsystem, apparatus, or device. The medium can be magnetic, optical,electronic, electromagnetic, infrared, or semiconductor system (orapparatus or device) or a propagation medium. The medium may include acomputer-readable storage medium such as a semiconductor or solid statememory, magnetic tape, a removable computer diskette, a random accessmemory (RAM), a read-only memory (ROM), a rigid magnetic disk and anoptical disk, etc.

Each computer program may be tangibly stored in a machine-readablestorage media or device (e.g., program memory or magnetic disk) readableby a general or special purpose programmable computer, for configuringand controlling operation of a computer when the storage media or deviceis read by the computer to perform the procedures described herein. Theinventive system may also be considered to be embodied in acomputer-readable storage medium, configured with a computer program,where the storage medium so configured causes a computer to operate in aspecific and predefined manner to perform the functions describedherein.

A data processing system suitable for storing and/or executing programcode may include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code to reduce the number of times code is retrieved frombulk storage during execution. Input/output or I/O devices (includingbut not limited to keyboards, displays, pointing devices, etc.) may becoupled to the system either directly or through intervening I/Ocontrollers.

Network adapters may also be coupled to the system to enable the dataprocessing system to become coupled to other data processing systems orremote printers or storage devices through intervening private or publicnetworks. Modems, cable modem and Ethernet cards are just a few of thecurrently available types of network adapters.

Referring now in detail to the figures in which like numerals representthe same or similar elements and initially to FIG. 1, a high-levelsystem/method for correcting defective products is illustrativelydepicted in accordance with one embodiment of the present principles.

In one embodiment, a manufacturing plant 10 manufactures products. Themanufactured products may be any physical product. The manufacturedproducts can sometimes be manufactured with defects. Thus, themanufactured products can be sent to a quality assurance system 20.

The quality assurance system 20 inspects the manufactured products toensure that the products live up to quality standards with respect toissues such as defects and anomalies in the products. Such defects caninclude visual defects, such as, e.g., cracks, holes, protrusions,discolorations, or any feature that is not part of the original designof the product. The quality assurance system 20 can include an automatedsystem for performing the inspections, such as, e.g., an automatedvisual system including a camera. However, other inspection systems canbe used, including, e.g., radar, infrared, ultrasound, or otherdetection methods. Based on results, the quality assurance system 20 canautomatically take steps to correct the product defects. Such steps caninclude, e.g., discarding the defecting product, automatically listingthe product as defective and requiring refurbishment, alerting anoperator to the defect, or any other corrective action.

According to an aspect of the present invention, the corrective actioncan include listing the product as defective and sending the productback to the manufacturing plant 10 to be recycled or refurbished.However, if the product is not defective, the product can be forwardedto a shipping and packaging system 30 to be sent to a customer.

Referring now to FIG. 2, a system/method for detecting and correctingdefects and anomalies is illustratively depicted in accordance with anembodiment of the present principles.

According to aspects of the present invention, defects and anomalies inan item are captured by an imaging device 100. The imaging device 100captures images of an item for analysis to facilitate recognizingdefects and anomalies. For example, the imaging device 100 can include,e.g., a camera device or multiple camera devices for capturing images ofeither the entire item or a portion of the item. Accordingly, multipleimaging devices 100 can be used to concurrently capture images of theitem from multiple perspectives. Thus, the entirety of the item, suchas, e.g., an entire surface, or for suitable imaging devices 100, anentirety of the depth (as can be the case for imaging devices 100including, e.g., magnetic resonance imaging (MRI) or computer aidedtomography, etc.) can be captured by the imaging device 100.Alternatively, a single imaging device 100 can be used, and the item andthe imaging device 100 can be moved relative to each other to captureimages from multiple perspectives by, e.g., conveyor, gimbal, or movingstage. The imaging device 100 can also include, e.g., an infraredsensor, a radar sensor, an ultrasound sensor, a light detection andranging (LIDAR) sensor, among others. Thus, the imaging device 100 cangenerate an image of the item for analysis.

The image can then be analyzed by an anomaly detection and taggingsystem 200. The anomaly detection and tagging system 200 process theimage to recognize the presence of any potential anomalies with the itemaccording to the image from the imaging device 100. Here, an anomaly isconsidered to be a feature of the item, as depicted in the image, thatdeviates from what is usually present at that location of the item. Forexample, the anomaly detection and tagging system 200 can compare theimage of the item to defectless items. Therefore, when the anomalydetection and tagging system 200 analyzes an image of an item that isdifferent from the a defectless item, the anomaly detection and taggingsystem 200 will determine where in the image and on the item thedifference is from the defectless item. The anomaly detection andtagging system 200 can then tag the feature of the item that isdifferent from the defectless item as a potential anomaly or defect.

Information regarding the tagged anomaly can then be provided to ananomaly correction system 300 to take corrective action. The correctiveaction can depend on the severity of the defects. For example, if theitem is a manufactured product, the production line may be stopped, orthe item may be discarded or otherwise removed for recycling orrefurbishment. A threshold may be used as well, such that if a thresholdnumber of anomalies are tagged, more drastic action may be taken. Forexample, if the item is a manufactured product, reaching a firstthreshold number of anomalies can result in the item being discarded,while reaching a second threshold can result in a halting of the entireproduction line. Alternatively, or in addition, the anomaly correctionsystem 300 can include a notification device to notify an operator ofthe item having tagged anomalies.

Referring now to FIG. 3, a system/method for detecting and correctingdefects and anomalies using a reconstructor and contrastor isillustratively depicted in accordance with the present principles.

According to aspects of the present invention, an image of an item takenby an imaging device 100 can be provided to both a reconstructor 220 anda contrastor 230. The reconstructor 220 can receive the image of theitem, e.g., in a memory or storage, and generate a reconstructed imageof the item that does not include any defects or anomalies such that thereconstructed image can be contrasted with the original image at thecontrastor 230.

The reconstructor 220 will receive the image of the item and reconstructthe image to remove any defects or anomalies. For example, thereconstructor can divide the image in multiple smaller portions of theimage, partially mask a region in each smaller portion, and reconstructeach portion. However, other methods of reconstructing the image arecontemplated. The reconstructor 220 can include, e.g., a processor toperform reconstruction on the image stored in a memory, such as, e.g., astorage, a random access memory (RAM), a buffer, or a cache, amongothers. The reconstructor 220 will, therefore, generate with thereconstructed image, a representation of the item that has no defects oranomalies. The reconstructed image can be stored or cached in a storagedevice, such as, e.g. a RAM, a buffer or a cache.

The reconstructed image will then be provided to the contrastor 230along with the original image from the imaging device 100. Thecontrastor 230 can then compare the reconstructed image with theoriginal image. Because the reconstructed image is a defectlessrepresentation of the item, any differences between the reconstructedimage and the original image will be detected by the contrastor 230.Similar to the reconstructor 220, the contrastor 230 can include, e.g.,a processor, to perform the contrasting, and a storage, such as, e.g., aRAM, buffer or cache, to temporarily or permanently store the originalimage and the reconstructed image.

The contrastor 230 will then provide data regarding the detecteddifference to an anomaly tagging device 240. The anomaly tagging device240 will use data regarding the locations of difference between thereconstructed image and the original image to identify anomalies. Thus,the anomaly tagging device 240 can generate a tagged image of the itemwith anomalies identified and tagged.

The tagged image can be used by the anomaly correction system 300 totake corrective action regarding the anomalies, as described above. Forexample, the anomaly correction system 300 can, e.g., automaticallydetermine that an item should be discarded if it has a certain number ofanomalies. Alternatively, the anomaly correction system 300 canautomatically determine that an item should be discarded if it has anyanomalies, or that the item can be sent back to manufacturing to berefurbished or recycled. The anomaly correction system 300 can even,e.g., automatically determine that an entire production line should bestopped if an item with anomalies is found, or if a certain number ofitems with anomalies is found. The anomaly correction system 300 canalso, e.g., notify an operator of the anomalies and providing theoperator with the tagged image.

Referring now to FIG. 4, a system/method for reconstructing an image fordetecting and correcting defects and anomalies is illustrativelydepicted in accordance with the present principles.

The reconstructor 220 can reconstruct the original image 221 of the itemto represent a defectless version of the item by extracting imagepatches with an image patch extractor 222. The image patch extractor 222can identify portions of the original image 221 to be reconstructed,such as, e.g., using a grid superimposed on the image. The image patchextractor 222 can then, using a processor, extract a series of imagepatches 222 a from the original image 221, where each image patch 222 ain the series is a different portion of the original image 221 accordingto the identified portions of the original image 221. To facilitatereconstruction of each image patch 222 a, a region 222 c in each imagepatch 222 a is blacked-out or otherwise masked to generate a partiallymasked image patch 222 b. In this way, data regarding features of theitem in the original image 221 is removed from the image patches 222 aso that reconstruction can be performed independent of any featurespresent in corresponding portions of the original image 221.

For each image in the series, a reconstruction module 223 canreconstruct the masked region 222 c of each partially masked image patch222 b. The reconstruction module 223 can, therefore, include, e.g., aprocessing device including a processor, and a storage device such as,e.g., a hard drive, solid state drive, a flash memory or a temporarymemory, such as, e.g., a RAM, a buffer, or a cache. To reconstruct themasked region 222 c, the reconstruction module 223 can utilize, e.g., anencoder-decoder arrangement stored in the storage device, or othersuitable neural network for reconstructing images.

According to aspects of the present invention, the reconstruction module223 is trained using defectless items of the type being analyzed. Thus,when the reconstruction module 223 reconstructs a masked region 222 cfor each partially masked image patch 222 b of the original image 221,it does so based on defectless training by predicting the contents ofthe masked region 222 c. As a result, the reconstructed portion willappear to be defectless in a reconstructed image patch. By training thereconstruction module 223 with defectless items, training images can beeasily found. Thus, training of the reconstruction module 223 is quickand efficient, with a large training set facilitating improved accuracy.Moreover, by reconstructing images to be defectless, types of defectsand anomalies do not need to be taken into account, thus reducing thecomplexity of the reconstruction, improving speed and efficiency of ananomaly detection and tagging system 200.

In embodiments of the present invention, the reconstruction module 223employs an encoder-decoder arrangement including an encoder 223 a anddecoder 223 b. Accordingly, the partially masked image patch 222 b witha masked region 222 c is provided to the encoder 223 a, which transformthe partially masked image patch 222 b with a hidden layer to a latentrepresentation in a feature space, such as, e.g., a multidimensionalfeature space vector. The hidden layer can include an activationfunction and a weight matrix. The encoder 223 a can include one or morehidden layers to arrive at an encoded representation, such as, e.g., themultidimensional feature space vector. In addition to masking featuresof the image patches 222 a, the encoder 423 a can be configured toreduce the dimensionality of the representation to further obfuscate anyfeatures in unmasked regions of the image patches 223, and thus reducethe risk of anomalies being present in a reconstructed representation ofthe image patches 223.

The encoded representation can then be decoded by the decoder 223 b togenerate a predicted image patch 224 a. Similar to the encoder 223 a,the decoder 223 b can use one or more hidden layers to transform theencoded representation to a representation corresponding to an outputimage by using an activation function and a weight matrix. Theactivation function and weight matrix of the decoder 223 b can be thesame or different from the activation function and weight matrix of theencoder 223 a.

Because the partially masked image patch 222 b includes a masked region222 c, the encoder 223 a encodes the partially masked image patch 222 bwithout any data related to any features of the item in the maskedregion 222 c. Thus, the features of the corresponding portion of theitem are not encoded in the multidimensional features space vector. As aresult, the decoder 223 b can then reconstruct the image patch 222 a bypredicting the masked region 222 c without any influence from featuresof a corresponding portion of the original image 221. Thus, a defectlessitem can be predicted corresponding to the masked region 222 c in anefficient manner.

Because the encoder 223 a and decoder 223 b have been trained withdefectless items, the decoder 223 b is trained to predict defectlessfeatures. Thus, the predicted image portion includes a reconstructedimage patch 224 a having no defects or anomalies, even if thecorresponding image patch 222 a of the original image 221 did havedefects or anomalies.

The reconstructed image patch 224 a can then be merged back into theoriginal image 221 with the image merging module 224. The image mergingmodule 224 can include, e.g., a processing device including a process,and a storage device such as, e.g., a hard drive, solid state drive, aflash memory or a temporary memory, such as, e.g., a RAM, a buffer, or acache. The reconstructed image patch 224 a will replace thecorresponding portion of the original image 221 such that the originalimage 221 becomes a reconstructed image 225. Alternatively, thereconstructed image patch 224 a can be stitched with other previouslyreconstructed image patches, independent of the original image. Eachextracted image patch 222 a will be reconstructed by the reconstructionmodule 223 and merged back into the image with the image merging module224. Thus, the reconstructed image 225 will be produced with everyidentified portion replaced with a reconstructed version of thatportion. As a result, the reconstructed image 225 will depict adefectless item. The reconstructed image 225 can then be stored, e.g.,in a storage device such as, e.g., a hard drive, solid state drive, aflash memory or a temporary memory, such as, e.g., a RAM, a buffer, or acache.

Referring now to FIG. 5, a system/method for contrasting a reconstructedimage and original image for detecting and correcting defects andanomalies is illustratively depicted in accordance with the presentprinciples.

A reconstructed image 225 can include an image having a number ofportions of an original image 221 reconstructed by a reconstructor 220.Thus, as discussed above, the reconstructed image 225 will depict anitem having any defects or anomalies removed by the reconstructionprocess. The reconstructed image 225 can be contrasted with the originalimage 221 by a contrastor 230 using, e.g., a processing device includinga processor and a storage device, such as, e.g., a hard drive, a solidstate drive, a flash memory, a RAM, a buffer or a cache. The contrastor230 can, e.g., determine a pixel-by-pixel difference between the imagesto produce an anomaly map 231. Areas of high contrast between the twoimages will result in a larger difference of pixels at that location.That difference can be mapped to a new image depicting thepixel-by-pixel difference, thus highlighting the anomalies in an anomalymap 231.

Referring now to FIG. 6, a system/method for training a reconstructorfor detecting and correcting defective items is illustratively depictedin accordance with the present principles.

A reconstructor can be trained to reconstruct images by training areconstruction learning module 423 with training images 421. Thetraining images 421 can each include an image of an item of a type to beanalyzed by an anomaly detection and tagging system, such as the anomalydetection and tagging system 200 discussed above. Each training image421 will be defectless, or in other words, “normal”. Thus, thereconstruction learning module 423 is training to reconstruct defectlessitem images. By training the reconstruction training module 423 withdefectless items, training images can be easily found. Thus, training ofthe reconstruction training module 423 is quick and efficient, with alarge training set facilitating improved accuracy. Moreover, byreconstructing images to be defectless, types of defects and anomaliesdo not need to be taken into account, thus reducing the complexity ofthe reconstruction, improving speed and efficiency of an anomalydetection and tagging system 200.

To train the reconstruction learning module 423, an image patchextractor 422 will extract patches 422 a of the training image 421. Theimage patch extractor 422, including a processing device having aprocessor, can identify portions of the training image 421 to bereconstructed, such as, e.g., using a grid superimposed on the image.The image patch extractor 422 can then extract a series of image patches422 a from the training image 421, where each image patch 422 a in theseries is a different portion of the training image 421 according to theidentified portions of the training image 421 and temporarily orpermanently store the image patches 422 a in a cache or a buffer.

A portion of each image patch 422 a can be blacked-out or otherwisemasked to form a masked region 422 c of the image patch 422 a togenerate a partially masked image patch 422 b. This masked region 422 ccontains no data regarding any features of a corresponding portion ofthe training image 421. Thus, reconstruction of each partially maskedimage patch 422 b can be performed independent of any features of thetraining image 421.

For each image in the series, a reconstruction learning module 423 canreconstruct the masked region 422 c of each partially masked image patch422 b. The reconstruction learning module 423 can, therefore, include,e.g., a processing device including a processor, and a storage devicesuch as, e.g., a hard drive, solid state drive, a flash memory or atemporary memory, such as, e.g., a RAM, a buffer, or a cache. Toreconstruct the masked region 422 c, the reconstruction learning module423 can utilize, e.g., an encoder-decoder arrangement stored in thestorage device, or other suitable neural network for reconstructingimages.

Accordingly, the partially masked image patch 422 b is provided to theencoder 423 a, which transforms the masked region 422 c with a hiddenlayer to a latent representation in a feature space, such as, e.g., amultidimensional feature space vector. The hidden layer can include anactivation function and a weight matrix. The encoder 423 a can includeone or more hidden layers to arrive at an encoded representation, suchas, e.g., the multidimensional feature space vector. Because thereconstruction learning module 423 is trained with partially maskedimages 422 b, it is unnecessary to reduce the dimensionality of themultidimensional features space vector to below that of an identityfunction. However, reducing the encoder 423 a can be configured toreduce the dimensionality of the representation to further obfuscate anyfeatures in unmasked regions of the image patches 423, and thus reducethe risk of anomalies being present in a reconstructed representation ofthe image patches 223.

The encoded representation can then be decoded by the decoder 423 b togenerate a predicted image patch 424 a. Similar to the encoder 423 a,the decoder 423 b can use one or more hidden layers to transform theencoded representation to a representation corresponding to an outputimage by using an activation function and a weight matrix. Theactivation function and weight matrix of the decoder 423 b can be thesame or different from the activation function and weight matrix of theencoder 423 a.

Because the reconstruction learning module 423 is trained with partiallymasked images 423, the reconstruction learning module 423 learnsreconstruction for image patches by predicting data independent from anypreexisting data in the masked portion 422 c. Complexity is, therefore,reduced for encoding and decoding, improving the speed and efficiency ofthe image reconstruction.

The predicted image can then be compared with the input image. An errorcan be determined according to difference between the input image andpredicted image using a loss function. The error can be backpropagate toeach hidden layer of the encoder 423 a and decoder 423 b to update theweight matrices at each layer using a suitable backpropagation process,such as, e.g., a gradient descent method, or a conjugate gradientmethod, among other backpropagation methods. This process is repeatedwith multiple training images. The training images will correspond toportions of images of defectless items of the type to be reconstructed.For example, the reconstruction learning module 423 can be trained withmanufactured products to reconstruct images of manufactured products.Thus, the encoder-decoder arrangement of the reconstruction learningmodule 423 will be trained to reconstruct images of defectless items.

While, the training can be performed as an independent process to trainthe reconstruction learning module 423 prior to implementing thereconstruction learning module 423 as a reconstructing module, such asthe reconstruction module 223 discussed above, the reconstruction modulecan be trained concurrently as a reconstruction learning module 423 withimplementing the reconstruction module to reconstruct product images.

Therefore, while reconstructing image portions of a product, areconstruction module, such as the reconstruction learning module 423can reconstruct the image patches 422 a. The reconstructed image patches424 a can then be merged back into the training image 421 with the imagemerging module 424. The reconstructed image patch 424 a will replace thecorresponding image patch 424 a of the training image 421 such that thetraining image 421 becomes a predicted reconstruction 425. Eachextracted image patch 422 a will be reconstructed by the reconstructionlearning module 423 and merged back into the image with the imagemerging module 424. Thus, the predicted reconstruction 425 can beproduced with every identified portion replaced with a reconstructedversion of that portion.

The reconstruction version can be contrasted with the original imageusing a process and system, such as the contrastor 230 and anomalytagging module 240 described above. If the original image is found to bedefect-free, the reconstructed portions can be used to determine anerror with, e.g., a loss function, and backpropagate that error to thehidden layers of the reconstruction learning module 423, as discussedabove. Thus, the reconstruction learning module 423 can continuously betrained with defect-free product images while concurrently determiningif a product has defects.

Referring now to FIG. 7, a high-level system/method for diagnosingmedical anomalies, in accordance with the present principles isillustratively depicted in accordance with the present principles.

In one embodiment, a medical scanning device 710 generates scans of aperson's anatomy, such as, e.g., an X-ray sensor, a magnetic resonanceimaging (MRI) device, a computed tomography (CT) scan, a positronemission tomography (PET) scan, optical image, or other scanning devicesuitable for medical diagnosis. The medical scanning device 710 can,therefore, generate an image of anatomy or physiology. A person'sanatomy or physiology can sometimes contain an anomaly that may indicatea disease or condition needing treatment. Thus, the image can beprovided to a medical anomaly detection system 720.

The medical anomaly detection system 720 will inspect the anatomy scansto determine if there are any anomalies. Such anomalies can include,e.g., physical signs and symptoms in anatomy and physiology thatindicate a medical abnormality including, e.g., a tumor, a blood clot, abroken bone, a dislocation, a fracture, among others. The medicalanomaly detection system 720 can determine that an abnormality exists bycomparing the scans to a scan that of normal anatomy and physiology. Forexample, e.g., the medical anomaly detection system 720 can include amachine learning system that is trained with images of medically normalpatient anatomy and physiology. By training the system with normalpatient scans, the medical anomaly detection system 720 can generate areconstruction of the anatomy scans that does not contain anyabnormalities, and compare the reconstructed version with the originalscans to identify differences. The locations of the differences willindicate an anomaly in anatomy or physiology. These differences can,therefore, be identified with the medical anomaly detection system 720.

The identified abnormalities can be communicated to a diagnosis system730, such as, e.g., a display, computer, or other notification device.Thus, the diagnosis system 730 can notify a doctor of the abnormalities.Therefore, the doctor can easily and quickly find anomalies in apatient's anatomy that may have otherwise gone unnoticed or undetected.Alternatively, the diagnosis system 730 can include a device forautomatically administering a medication. For example, where the medicalscanning device 710 is an X-ray device, an anomaly may correspond to abroken bone. Thus, the diagnosis system 730 can, e.g., automaticallyadminister a pain killer in response to tagging a bone related anomaly.

Referring now to FIG. 8, a system/method for detecting defects andanomalies with a reconstructor and contrastor is illustratively depictedin accordance with the present principles.

At block 801, extract an image patch from a location on an originalimage of an item and partially mask the image patch.

The original image can be captured with an imaging device, such as,e.g., a camera, CCD, infrared sensor, LIDAR sensor, or other device forcapturing images. The original image will be an image of the item, suchas, e.g., a manufactured product, or a part of anatomy, among otheritems. Occasionally, the item will include defects and anomalies thatundesired, such as, e.g., cracks, burrs, protrusions, dents, etc. Theimage device will capture these anomalies and defects.

Portions of the original image can then be identified to bereconstructed, such as, e.g., using a grid superimposed on the image.The portions are then extracted as a series of image patches from theoriginal image, where each image patch in the series is a differentportion of the original image according to the identified portions ofthe original image. To hide an area of the portions so that the area canbe reconstructed, the area can be masked by, e.g., blacking-out, hiding,or otherwise removing the area from subsequent processing.

At block 802, encode the partially masked image patch by transformingthe partially masked image patch to a feature space vector using one ormore hidden layers of an encoder.

In embodiments of the present invention, the image patch is provided toan encoder, which transform the image patch with a hidden layer to afeature space vector. The hidden layer can include an activationfunction and a weight matrix. The encoder can include one or more hiddenlayers to arrive at the feature space vector.

At block 803, reconstruct the partially masked image patch of the itemby decoding the feature space vector into a reconstructed patch usingone or more hidden layers of a decoder.

The encoded representation can then be decoded by a decoder to generatea predicted image patch that returns the encoded representation back toa representation having the original number of dimensions of the inputimage patch. Similar to the encoder, the decoder can use one or morehidden layers to transform the encoded representation to arepresentation corresponding to an output image patch by using anactivation function and a weight matrix. The activation function andweight matrix of the decoder can be the same or different from theactivation function and weight matrix of the encoder.

Because the encoder encodes a partially masked image portion, thedecoder predicts a masked region of the partially masked image portion.Thus, contents, including, e.g., physical or visible features of imagepatch used as the input image patch, are predicted during decoding bythe decoder without influence from any features of the original image ofthe item. As a result, the output representation including the predictedimage patch reconstructs the image patch according to the weight matrixand activation function.

At block 804, merge the reconstructed patch into the location on theoriginal image to generate a reconstructed image.

The reconstructed image patch can then be merged back into the originalimage. The reconstructed image patch will replace the correspondingportion of the original image such that the original image becomes areconstructed image. Alternatively, the reconstructed image patch can bestitched with other previously reconstructed image patches, independentof the original image.

Each extracted image portion will be reconstructed and merged back intothe image. Thus, the reconstructed image will be produced with everyextracted patch replaced with a corresponding reconstructed patch. As aresult, the reconstructed image will depict a defectless item. Thereconstructed image can then be stored, e.g., in a storage device suchas, e.g., a hard drive, solid state drive, a flash memory or a temporarymemory, such as, e.g., a RAM, a buffer, or a cache.

At block 805, contrast the reconstructed image with the original togenerate an anomaly map that indicates anomalies at locations ofdifferences between the reconstructed image and the original image.

To determine anomalies and defects in the item, the reconstructed imagecan then be compared to the original image with a contrastor. Thecontrastor can compare the reconstructed image with the original imageby, e.g., performing a pixel-by-pixel difference between the images.However, other contrasting methods are contemplated. Because of thecontrasting, the differences between the reconstructed image andoriginal image can be mapped in an anomaly map that represents a degreeof difference between the reconstructed image and the original image ateach location. The anomaly map can take the form of, e.g., a visualrepresentation, such as an image, or a matrix representation, or by anyother suitable representation. As a result, the anomaly map can take theform of separate representation of the item, or it can be overlaid ontothe original image to provide both a representation of the actual item,as well as a representation of the anomalies.

At block 806, tag anomalies on the anomaly map to indicate possibleanomalies in the item corresponding to the differences between thereconstructed image and the original image.

The anomaly map can then be used to identify anomalies by tagging areasof greatest difference between the reconstructed image and the originalimage corresponding to the results in the anomaly map. Tagged areas canbe determined according to, e.g., an anomaly threshold value thatrepresents degree of difference between the reconstructed image and theoriginal image. If an area has a difference as represented in theanomaly of greater than the anomaly threshold, then that area can betagged as containing an anomaly on a corresponding location of the item.Thus, item defects and anomalies can be identified. The anomaly tags canbe applied to, e.g., the anomaly map, the original image, or both, or asa separate representation, such as, e.g., a list with coordinates.

At block 807, automatically correct the anomalies.

In response to the anomaly map and the anomaly tags, corrective actioncan be taken. For example, a product having anomalies indicating defectscan be automatically removed from a production line, or the entireproduction line can be automatically stopped. As another possiblecorrective action, a notification can be provided to an operator via anotification system, such as, e.g., a display or audible alert, suchthat the operator can take an appropriate action.

The foregoing is to be understood as being in every respect illustrativeand exemplary, but not restrictive, and the scope of the inventiondisclosed herein is not to be determined from the Detailed Description,but rather from the claims as interpreted according to the full breadthpermitted by the patent laws. It is to be understood that theembodiments shown and described herein are only illustrative of theprinciples of the present invention and that those skilled in the artmay implement various modifications without departing from the scope andspirit of the invention. Those skilled in the art could implementvarious other feature combinations without departing from the scope andspirit of the invention. Having thus described aspects of the invention,with the details and particularity required by the patent laws, what isclaimed and desired protected by Letters Patent is set forth in theappended claims.

What is claimed is:
 1. A method for detecting and correcting defectiveproducts, the method comprising: capturing at least one image of aproduct with at least one image sensor to generate an original image ofthe product; encoding, with an encoder, each of at least one partlymasked portion of an image extracted from the original image to generatea feature space vector; decoding, with a decoder, the feature spacevector to reconstruct the at least one masked portion of the image intoa corresponding at least one reconstructed portion by predictingdefect-free structural features in each of the at least one maskedportion according to hidden layers trained to predict defect-freeproducts; merging each of the at least one reconstructed portion into areconstructed image of a defect-free representation of the product; andcommunicating the reconstructed image to a contrastor to detectanomalies indicating defects in the product.
 2. The method as recited inclaim 1, further including contrasting, with a contrastor, the originalimage with the reconstructed image to generate an anomaly map indicatinglocations of difference between the original image and the reconstructedimage.
 3. The method as recited in claim 2, wherein the contrastordetermines a pixel-by-pixel difference between the reconstructed imageand the original image.
 4. The method as recited in claim 2, wherein theanomaly map includes a matrix of difference values corresponding todifferences between the reconstructed image and the original image at aplurality of locations on the original image.
 5. The method as recitedin claim 2, wherein the anomaly map includes an image depicting adifference between the reconstructed image and the original image at aplurality of locations on the original image.
 6. The method as recitedin claim 1, further including tagging, with an anomaly tagging device,locations of difference as anomalies corresponding to defects on theproduct.
 7. The method as recited in claim 1, further includingautomatically discarding the product according to the detectedanomalies.
 8. The method as recited in claim 1, further including animage patch extractor to extract at least one image patch and partiallymask each of the at least one image patch to generate the at least onemasked portion of the original image.
 9. The method as recited in claim8, wherein the image patch extractor imposes a grid across the originalimage, with each area defined by the grid corresponds to a portion to beextracted.
 10. The method as recited in claim 1, further includingnotifying an operator, with a notification device, of the anomalies. 11.The method as recited in claim 10, wherein the notification deviceincludes a display for displaying the original image of the product withtags corresponding to the anomalies.
 12. A method for detecting andcorrecting defective products, the method comprising: extracting, withan image patch extractor, portions of an original image of a product andpartially masking each portion to form at least one masked portion;reconstructing, with a reconstructor, the original image, including:encoding, with an encoder, each of at least one partly masked portion ofan image extracted from the original image to generate a feature spacevector; decoding, with a decoder, the feature space vector toreconstruct the at least one masked portion of the image into acorresponding at least one reconstructed portion by predictingdefect-free structural features in each of the at least one maskedportion according to hidden layers trained to predict defect-freeproducts; merging, with an image merging module, the at least onereconstructed portion into the original image to generate areconstructed image; contrasting, with a contrastor, the original imageof the product with the reconstructed image to generate an anomaly mapindicating locations of difference between the original image and thereconstructed image; tagging, with an anomaly tagging device, thelocations of difference as anomalies corresponding to defects on theproduct; and notifying, automatically via a display, an operator of theanomalies.
 13. The method as recited in claim 12, wherein the imagepatch extractor imposes a grid across the original image, with each areadefined by the grid corresponds to a portion to be extracted.
 14. Themethod as recited in claim 12, further including blacking out an area ofthe portions to be reconstructed to generate the masked portions. 15.The method as recited in claim 12, wherein the contrastor determines apixel-by-pixel difference between the reconstructed image and theoriginal image.
 16. The method as recited in claim 12, wherein thedisplay displays the original image of the product with tagscorresponding to the anomalies.
 17. A system for detecting andcorrecting defective products, the system including: at least one imagesensor for capturing at least one image of a product to generate anoriginal image of the product; a reconstructor to reconstruct anoriginal image of a product by reconstructing partially masked portionsof the original image to be a defectless representation of correspondingportions of the product, the reconstructor including: an encoder toencode each of at least one partially masked portion of an imageextracted from the original image to generate a feature space vector; adecoder to decode the feature space vector to reconstruct the at leastone portion of the image into a corresponding at least one reconstructedportion by predicting defect-free structural features in each of the atleast one portion according to hidden layers trained to predictdefect-free products; an image merging module to merge each of the atleast one reconstructed portion into a reconstructed image of adefect-free representation of the product; and an anomaly correctionsystem to automatically discard the product according to the taggedanomalies.
 18. The system as recited in claim 17, further including acontrastor to contrast the original image of the product with thereconstructed image to generate an anomaly map indicating locations ofdifference between the original image and the reconstructed image. 19.The system as recited in claim 17, further including an anomaly taggingdevice to tag the locations of difference as anomalies corresponding todefects on the product.
 20. The system as recited in claim 17, furtherincluding a display for displaying the original image of the productwith tags corresponding to the anomalies.